Method and apparatus for self-relative tracking using magnetic tracking

ABSTRACT

A method and apparatus for self-relative body tracking in virtual reality systems using magnetic tracking is disclosed, which allows more accurate tracking of a user&#39;s body relative to the user&#39;s field of vision. A head-mounted magnetic tracking (HMMT) system is used, in which other parts of a user&#39;s body are tracked relative to the HMD on the user&#39;s head, rather than relative to a base station. This results in less distortion of the magnetic field used for tracking and thus allows for more accurate determination of the position and orientation of a user&#39;s body parts relative to the user&#39;s field of vision, so that a more accurate avatar may be presented on the HMD to the user. This allows the avatar of the user&#39;s body part to be shown in a location that corresponds closely to its physical position. In an alternative embodiment, multiple portions of the user&#39;s body may be tracked.

This application claims priority to Nonprovisional patent applicationSer. No. 14/797,734, filed Jul. 13, 2015, now issued as U.S. Pat. No.10,162,177 on Dec. 25, 2018, and to Provisional Application No.62/023,756, filed Jul. 11, 2014, each of which is incorporated byreference herein in its entirety.

FIELD OF THE INVENTION

The present invention relates generally to virtual reality devices, andmore particularly to trackers for use in a virtual reality context.

BACKGROUND OF THE INVENTION

Virtual reality is often used to describe a wide variety of applicationscommonly associated with immersive, highly visual, computer-simulatedenvironments that can simulate a user's physical presence in places inthe real world or imagined worlds. While virtual reality can recreate anumber of sensory experiences, the senses most commonly used today tocreate virtual reality appear to be sight and sound.

One method of presenting a virtual world to a user that is commonlyidentified with virtual reality and presently in use is through the useof a visor or helmet containing a video display which encompasses partor all of a user's field of view and presents computer generated imagesrepresenting the virtual reality environment, or “virtual world,” to theuser. Such a device is often referred to as a head-mounted display, orHMD. One type of HMD presently available is the Oculus Rift from OculusVR, now owned by Facebook.

Typically a HMD covers the user's eyes, so that the user sees only thevirtual world while wearing the HMD and is thus unable to see the actualphysical world around the user while in the virtual world. In thephysical world a person is normally able to see his or her arms, legsand torso when they are within the person's field of vision, but a userof a HMD which covers the entire field of the user's vision is unable todo so. To enhance the realism of the virtual world, in some cases it isthus desirable that the user see in the virtual world an avatar of abody, or at least some portions thereof, which correspond to the user'sphysical sensations of his or her own body.

In some systems, such an avatar may be created by displaying virtualbody parts such as arms, legs and torso in a position and orientationsimilar to how such body parts might ordinarily appear in the user'sfield of vision in the physical world. In order to display such virtualbody parts in locations that at least approximate where the user's bodyparts are physically located, it is desirable to know the position andorientation of such parts of the user's body. The more precisely theactual physical position and orientation of a user's body part relativeto the user's field of vision may be determined, the more accurately avirtual world avatar of a user may be portrayed in the user's field ofvision on an HMD.

One known way of determining the position and orientation of a user'shead is by attaching to the HMD a tracker which is able to sense amagnetic field generated from a local base station, video game consoleor other apparatus. The tracker provides information about the sensedmagnetic field to a processor, which derives the position andorientation of the tracker, and thus the HMD, relative to the basestation from such information.

It is also known to place such a tracker in a handheld controller, or toattach it to one or more of the user's limbs, which similarly allows theposition and orientation of the controller or limbs to be derived by theprocessor. The processor may then extrapolate the position of the user'shand holding the controller from the position of the controller, anddisplay an avatar of the user's hand to the user on the HMD based uponsuch extrapolation.

However, there are a number of factors that limit the accuracy of basestation type systems. The strength of the magnetic field between atracker and a base station suffers from attenuation over distance, whichdecreases accuracy in the detection of the magnetic field by thetracker. Objects located between the tracker and the base station maycause distortion in the magnetic field, particularly if those objectscontain ferrous materials. Conducting surfaces in the environment mayalso contain eddy currents induced by the source magnetic field which inturn generate secondary magnetic fields that interfere with the abilityof sensors to detect the source magnetic field. Still further, if atracker is located on a handheld controller, the extrapolation of theposition of the user's hand may be inaccurate, as the position of thetracker on the handheld controller may not precisely correspond to theposition of the user's hand depending upon the size of the controllerand where on the controller the tracker is located.

Thus, a base station type system may not be able to track desiredportions of a user's body with sufficient accuracy to generate an avatarof those portions of the body in their precise positions relative to theuser's field of vision such that the user sees the avatar of his or herbody portions to be in the same locations in which they are physicallyfelt by the user.

SUMMARY OF THE INVENTION

A system and method is disclosed which provides for self-relative bodytracking in virtual reality systems using magnetic tracking.

One embodiment discloses a system for use with a head-mounted display(HMD), the HMD covering a user's field of vision, comprising: a magnetictracking system comprising a plurality of source magnetic coilsconfigured to generate a magnetic field and a plurality of sensormagnetic coils configured to sense the magnetic field, one of thepluralities of magnetic coils configured to be attached to the HMD andthe other plurality of magnetic coils configured to be attached to alimb of the user; and a processor configured to: generate and outputinstructions to the HMD to display on the HMD a virtual world; determinethe position and orientation of the users' limb relative to the HMDusing information from the magnetic tracking system; and generate andoutput instructions to the HMD to display on the HMD a virtual limb inthe virtual world, the virtual limb in an apparent position andorientation corresponding to the actual physical position andorientation of the user's limb.

Another embodiment discloses a method of providing an avatar of the bodyof a user wearing a head-mounted display (HMD) to the user, the HMDcovering the user's field of vision, the method comprising: attachingtwo pluralities of magnetic coils to the user, one plurality of coils tothe HMD and the other plurality of magnetic coils to a limb of the user;generating a magnetic field from one of the pluralities of magneticcoils; generating magnetic sensor data by sensing the magnetic fieldusing the other plurality of coils; generating and outputting, by aprocessor, instructions to the HMD to display on the HMD a virtualworld; determining, by the processor, the position and orientation ofthe user's limb relative to the HMD using the magnetic sensor data; andgenerating and outputting, by the processor, instructions to the HMD todisplay on the HMD a virtual limb in the virtual world, the virtual limbin an apparent position and orientation corresponding to actual physicalposition and orientation of the user's limb.

Still another embodiment discloses a non-transitory computer-readablemedium having embodied thereon instructions for causing a computingdevice to execute a method of providing an avatar of the body of a userwearing a head-mounted display (HMD) to the user, the HMD covering theuser's field of vision, the method comprising: attaching two pluralitiesof magnetic coils to the user, one plurality of coils to the HMD and theother plurality of magnetic coils to a limb of the user; generating amagnetic field from one of the pluralities of magnetic coils; generatingmagnetic sensor data by sensing the magnetic field using the otherplurality of coils; generating and outputting, by a processor,instructions to the HMD to display on the HMD a virtual world;determining, by the processor, the position and orientation of theuser's limb relative to the HMD using the magnetic sensor data; andgenerating and outputting, by the processor, instructions to the HMD todisplay on the HMD a virtual limb in the virtual world, the virtual limbin an apparent position and orientation corresponding to actual physicalposition and orientation of the user's limb.

A further embodiment discloses a system for capturing the motion of alimb of a user relative to the user's torso, comprising: a magnetictracking system comprising a plurality of source magnetic coilsconfigured to generate a magnetic field and a plurality of sensormagnetic coils configured to sense the magnetic field, one of thepluralities of magnetic coils configured to be attached to the user'storso and the other plurality of magnetic coils configured to beattached to a limb of the user; and a processor configured to determinethe position and orientation of the user's limb relative to the user'storso using information from the magnetic tracking system.

Another embodiment discloses a system for displaying the motion of firstuser to a second user, comprising: a magnetic tracking system comprisinga plurality of source magnetic coils configured to generate a magneticfield and a plurality of sensor magnetic coils configured to sense themagnetic field, one of the pluralities of magnetic coils configured tobe attached to the first user's torso and the other plurality ofmagnetic coils configured to be attached to a limb of the first user;and a processor configured to: determine the position and orientation ofthe first user's limb relative to the first user's torso usinginformation from the magnetic tracking system; and generate and outputinstructions to display to the second user on a display device an avatarof the first user, the avatar of the first user having a virtual torsoand a virtual limb in an apparent relative position and orientationcorresponding to the actual relative physical position and orientationof the first user's torso and limb.

A further embodiment discloses a system for use with an augmentedreality display (ARD), the ARD overlaying a user's field of vision,comprising: a magnetic tracking system comprising a plurality of sourcemagnetic coils configured to generate a magnetic field and a pluralityof sensor magnetic coils configured to sense the magnetic field, one ofthe pluralities of magnetic coils configured to be attached to the ARDand the other plurality of magnetic coils configured to be attached to aphysical object; and a processor configured to: determine the positionand orientation of the physical object relative to the ARD usinginformation from the magnetic tracking system; and generate and outputinstructions to the ARD to display information on the ARD based upon theposition and orientation of the physical object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are illustrations of a user wearing a head mounteddisplay (HMD).

FIG. 2 is an illustration of a user wearing multiple trackers on theuser's body.

FIG. 3 is an illustration of a generated virtual world including avatarsof a user's hands.

FIG. 4 is a flowchart of a method of generating an avatar of one or moreof a user's body parts in a generated virtual world.

DETAILED DESCRIPTION OF THE INVENTION

A method and apparatus for self-relative body tracking in virtualreality systems using magnetic tracking is disclosed, which allows moreaccurate tracking of a user's body relative to the user's field ofvision. A head-mounted magnetic tracking (HMMT) system is used, in whichother parts of a user's body are tracked relative to the HMD on theuser's head, rather than relative to a base station. This results inless distortion of the magnetic field used for tracking and thus allowsfor more accurate determination of the position and orientation of auser's body parts relative to the user's field of vision, so that a moreaccurate avatar may be presented on the HMD to the user.

In one embodiment, in a system or method for use with a HMD, a magneticfield source comprising a plurality of source magnetic coils whichgenerate a magnetic field is attached to the HMD on the user's head, anda tracker comprising a plurality of sensor magnetic coils which sensethat magnetic field is attached to a limb of the user. As is known inthe art, a processor uses data from the tracker to determine therelative position of the user's limb and the HMD, and generates anavatar of the user's limb that is displayed on the HMD.

The closeness of the magnetic field source and the tracker allows formore accurate determination of the position and orientation of thetracker. This in turn allows the avatar of the user's limb to be shownin a location that corresponds very closely to the physical position ofthe user's limb, so that the user sees his or her limb in the virtualworld in a position that is close to the position in which the userfeels the limb to be.

In an alternative embodiment, there may be multiple pluralities ofsensor coils, each plurality of sensor coils located on a differentportion of the user's body. For example, there may be separate sensorcoils on each hand of the user, and each foot. In some cases it may evenbe desirable to have a plurality of sensor coils located on the user'storso. The processor may then generate an avatar to be displayed on theHMD, which each limb (or the torso) of the avatar shown in a positioncorresponding to the actual physical position of the user's actual limb.

FIG. 1A shows a user 102 in the real world wearing a HMD 106; HMD 106covers the user's eyes and presents the user with a representation of avirtual world, while preventing the user from seeing the real, orphysical, world. Attached to or incorporated in the HMD 106 is amagnetic field source 108 comprising a plurality of source magneticcoils, which generates a magnetic field.

FIG. 1A also shows the user 102 holding a controller 110 which the usermay pick up with his or her hand(s), and which allows the user toperform various functions in the virtual world. Various devices suitablefor use as controller 110 will also be known to those of skill in theart. The controller 110 is considered to be attached to the user's handwhen the user holds it.

Attached to controller 110 is also a tracker 114, comprising a pluralityof sensor magnetic coils. The sensor magnetic coils in tracker 114 sensethe magnetic field emanating from magnetic field source 108, and providedata about the sensed magnetic field from which a processor can derivethe position and orientation of tracker 114 relative to the magneticfield source 108, and thus relative to the HMD 106. In embodiments,tracker 114 may be contained within the body of a controller 110. Instill other embodiments, tracker 114 may be directly attached to theuser's limb by means of, for example, a strap of elastic material or ahook and loop material such as Velcro. One of skill in the art willappreciate that there are other ways in which tracker 114 may beattached to a user as well.

Such trackers, and the processors which determine the position andorientation of such trackers, are well known to those of skill in theart. One such example, U.S. Pat. No. 4,737,794, teaches a “method andapparatus for determining remote object orientation and position with anelectromagnetic coupling.” It shows a plurality of radiating antennaslocated at a source to provide a plurality of electromagnetic fieldsspanning three dimensional space and defining a source referencecoordinate frame, and a plurality of receiving antennas located on anobject to be tracked for receiving that field. A processor receives theoutputs from the receiving antennas and converts the received componentsof the transmitted electromagnetic fields into remote object positionand orientation relative to the source reference coordinate frame. Theantennas may be, for example, dipole antennas, loop antennas, or coilantennas.

Attaching magnetic field source 108 to HMD 106 allows for an exact knownposition and orientation offset between magnetic field source 108 andthe display screen of HMD 106. By applying this offset to themeasurements from tracker 114, the position and orientation of tracker114 relative to the display screen of HMD 106 can also be determinedwith great accuracy, and thus where tracker 114, and the user's hand, iswithin the user's field of view.

The positions of the magnetic field source 108 and tracker 114 in FIG.1A may be reversed if desired for convenience, i.e., magnetic fieldsource 108 may be attached to the user's hand and tracker 114 attachedto HMD 106. What is important is that one of magnetic field source 108and tracker 114 is located on HMD 106 and the other on a part of theuser's body so as to track the relative position of HMD 106 and theuser's body part.

FIG. 1B is a representation of the user 102 of FIG. 1A as the user 102sees the virtual world, showing the field of view 112 that may bepresented to the user by the HMD 106. As will be known to those of skillin the art, a processor (not shown) generates instructions that causeHMD 106 to present a view to user 102 that is appropriate given theposition and orientation of user 102's head and any actions taken withcontroller 110, so that what is within field of view 112 will change asuser 102 moves his or her head and uses controller 110.

In the illustration of FIG. 1B, the user is seeing the virtual world aspresented on HMD 106. In FIG. 1B the user also sees a virtual object 116in his or her hand, here a virtual gun, rather than the controller 110that the user is actually holding as in FIG. 1A. The user has presumablyused the controller to interact with the virtual world to “pick up” thevirtual gun 116. Tracker 114 remains on controller 110 but in someembodiments will not be visible in the virtual world.

Tracker 114 is attached to controller 110, and thus magnetic fieldsource 108 is closer to tracker 114 than a base station would typicallybe. For this reason, tracker 114 may sense the magnetic field moreaccurately than it would be likely to sense the magnetic field from abase station, and thus the determination of the position and orientationof tracker 114, and resultantly the user's hand, is likely to be moreaccurate than the case of a base station. In addition, the closeness ofthe magnetic field source 108 and tracker 114 make it likely that therewill not be any objects between them, or any nearby surfaces which maycontain eddy currents, which could interfere with the magnetic field.

The magnetic field source 108 may be attached to the HMD in a variety ofways and a variety of configurations, as long as the relative positionand orientation between the magnetic field source 108 and the displayscreen of HMD 106 is fixed. In the case of the Oculus Rift, such a fixedrelative position may be obtained by including in magnetic field source108 a male USB plug which is inserted into a USB host port on top of theHMD. This also provides power and data to magnetic field source 108 andany wireless data channels to tracker 114.

In some embodiments, trackers may be placed on more locations on thebody of a user, as shown in FIG. 2. As illustrated, user 102 now hasfive different trackers attached to various portions of the body.Tracker 202 is attached to the right hand or arm of user 102, tracker204 to the left hand or arm, tracker 206 to the right foot or leg,tracker 208 to the left foot or leg, and tracker 210 to the waist ortorso of user 102. User 102 again wears an HMD with a magnetic fieldsource attached, such as HMD 106 and magnetic field source 108 of FIG.1, although these are not shown in FIG. 2.

As with tracker 114 in FIG. 1, each of trackers 202, 204, 206, 208 and210 in FIG. 2 senses the magnetic field from magnetic field source 108attached to the HMD 106 worn by user 102. A processor, not shown, againdetermines the position and orientation of each tracker from dataprovided by the tracker about the sensed magnetic field. Again thetrackers are close to the magnetic field source, so that the positionand orientation of each tracker may be accurately determined.

More or fewer trackers may be utilized in various embodiments. Thenumber of trackers which is desirable may depend upon the particularapplication that is being used and the virtual world to be presented onthe HMD for that application. One of skill in the art will be able tomake a determination of how many trackers are appropriate, and where ona user's body they should be located in order to create an appropriateavatar for the application.

As discussed above, the position of magnetic field source 108 may beswitched with any one of trackers 202 to 210. Again, what is importantis that one device of magnetic field source 108 and trackers 202 to 210is attached to HMD 106, and the remaining devices are attached to partsof the user's body, so that the relative positions of HMD 106 and theuser's body parts may be accurately determined. No matter how the usermoves his or her head, all tracked hardware will visually appear to bein the correct location.

By mounting magnetic field source 108 and any trackers on either HMD 106or the user's body, the distance between magnetic field source 106 andall of the trackers is bounded by the dimensions of the user's body.This limited distance reduces signal attenuation, and also the effectsof any secondary magnetic fields in the vicinity of the user, resultingin increased accuracy.

As in the discussion above, the offset between magnetic field source 108and the screen display of HMD 106 is applied to all of the trackers inuse to determine their positions relative to the screen and thus wherethe body parts to which the trackers are attached lie in the user'sfield of vision. This offset between magnetic field source 108 and thescreen display of HMD 106 can be pre-determined from the geometry of themounting hardware that attaches magnetic field source 108 to HMD 106, orit may be measured and manually input by the user if a particular HMD106 does not provide support for a fixed mount.

One way to pre-determine this offset would be to build magnetic fieldsource 108 directly into HMD 106 that contains the display, so that theoffset is determined by the physical design of the unit. The offset canalternatively be constrained by requiring the user to mount magneticfield source 108 to the head strap of HMD 106, or by providing a modularbracket that securely connects them. Magnetic field source 108 couldalso be attached to other worn accessories or devices that are placed onthe head, such as headphones or a hat.

When magnetic field source 108 is attached by a rigid fixture to HMD106, a calibration process can be implemented to measure theattenuation/amplification caused by HMD 108 in each ordination of thecoil(s) of magnetic field source 108 to each of the operatingfrequencies, and a compensation matrix for this condition can begenerated. The system will use this compensation matrix when magneticfield source 108 is attached to HMD 106 to mitigate any distortioncaused by electrical components.

It may also be possible to isolate the contribution of HMD 108 to thesignal biases from per-component variance that occur naturally from onecoil to another to creating a model that can output a new set ofcoefficients that are correct for any combination of individual coilsand any new HMD introduced to the market. This would allow end users toperform software updates to enable the tracking system to functionoptimally with new display hardware, without requiring additionalper-coil calibration testing in a controlled environment.

It will be appreciated that because tracking is not performed relativeto a base station which has a fixed position in the physical world, theabsolute position of any trackers attached to the user's body are notfully determined by the HMMT. Rather, only the relative positions of thetrackers and magnetic field source 108 are determined. If it isdesirable to know the actual position of the HMD, the HMD, or the user'shead, may be tracked relative to a fixed physical location, such as abase station, by a separate system.

If it is desired to know the absolute position of the user, or toreproduce some other physical object in the virtual world, such as atable or floor, in some embodiments a tracker may be placed on such anobject, which can then also be used as a reference frame to determinethe absolute position and orientation of magnetic field source 108 andthus the absolute position and orientation of the user's head.Alternatively, in some embodiments, the orientation of the user's headmay be determined by the use of inertial sensors so that an appropriateorientation for the virtual world to be shown on screen display of HMD106 may be selected.

As above, in the virtual world, the position and orientation of eachtracker is used to determine the position and orientation of acorresponding virtual body part which is then shown in a visuallycorrect position on the screen display of HMD 106. Thus, in FIG. 1above, the position and orientation of user 102's hands is known fromthe position and orientation of tracker 114. However, the actual pose ofthe user's hands grasping controller 110 may not be the pose that it isdesired to be displayed, i.e., the grip on controller 110 may notcorrespond to a grip that would normally be used on virtual object 116.

In this case, the pose of the user's hand to be displayed may beinferred based on how controller 110 is designed to be gripped usinginverse kinematics. Inverse kinematics is a known technique foranimating 3D characters in which a 3D model is posed like a mannequin inorder to position a particular body part, such as a hand, at a desiredlocation in space. The virtual hand may be located at a position in thevirtual world corresponding to the position of the tracker on the user'sphysical hand, and the joints of the virtual hand rotated to approximatethe pose in which the hand would be expected to be seen in a particularapplication, rather than the actual pose of the hand gripping controller110.

Thus, FIG. 3 shows one embodiment of what a user such as user 102 mightsee in the virtual world displayed in the HMD 106 in one embodiment. Inthe illustrated embodiment, the processor has generated instructions tothe display of HMD 106 to create a picture of a virtual world as if theuser is looking toward a back wall 302, and also displays side walls 304and floor 306.

In FIG. 3 it is assumed that trackers are attached to the user's hands,and that the user's field of view of the virtual world includes a regioncorresponding to the region of the physical world in which the user'shands are physically located. (There may be other trackers attached, forexample, to the user's legs or torso, but, if so, the user is notpresently looking in the direction of his or her legs or torso.)

Since the processor is able to determine from information received fromthe trackers attached to the user's hands where the hands are located,the processor may display both virtual hands 308 in apparent locationswithin the virtual world that correspond to where the user's physicalhands are actually located. This heightens the illusion of realitypresented in the virtual world, as the user sees the avatar of his orher hands in a location corresponding to the location in which the userfeels his or her actual physical hands to be, even though the usercannot actually see the physical hands through the HMD.

Even though the user 102 in FIG. 1 is actually gripping a controller110, in FIG. 3 the virtual hands 308 are shown in an inferred pose, hereopen hands, rather than in a gripping pose.

The processor can continue to generate instructions to the display inthe HMD to present the user with changes in the virtual world based uponactions that the user has performed with a controller, such ascontroller 110 in FIG. 1. In a conventional video game, for example, auser may use a controller to have a character on a television screen“pick up” a weapon displayed on the screen. Similarly, in a virtualreality setting the processor can cause a HMD to display a virtualweapon while the user is in a virtual world.

Then, upon the user's extending his or her physical hand(s) in such away as to cause the corresponding virtual hand(s) to “reach for” thedisplayed virtual weapon and the user's activating a proper command(e.g., by pressing a button on the controller), the processor can causethe HMD to display the user's virtual hand(s) 308 as “picking up” andholding that virtual weapon. Virtual hands 308 may continue to be shownin an apparent location corresponding closely to the actual location ofthe user's hands with respect to the user's field of vision and in aposition indicative of holding the weapon.

FIG. 4 is a flowchart of a method implemented by a processor that mightresult in the virtual world display(s) described herein according to oneembodiment. It is assumed that, as above, a user wears a HMD having adisplay, a processor provides instructions that direct what appears onthe display, and that there is a magnetic field source having magneticcoils configured to generate a magnetic field and a tracker havingmagnetic coils configured to sense the magnetic field and provide sensordata. It is also assumed that the processor is able to determine theposition and orientation of the tracker relative to the magnetic fieldsource from such sensor data.

At step 402, a magnetic tracking system comprising two pluralities ofmagnetic coils is attached to the user. As above, one plurality ofmagnetic coils is configured to generate a magnetic field and the otherplurality of magnetic coils is configured to sense the magnetic fieldand generate sensor data. One plurality of magnetic coils is attached tothe HMD, and the other plurality of magnetic coils is attached to a limbof the user.

As shown in FIG. 1 above, the magnetic coils that are configured togenerate a magnetic field (magnetic source 108) are attached to the HMD106, and the other magnetic coils that are configured to sense themagnetic field (tracker 114) are attached to the user's hand. Again,these positions may be reversed, so that the magnetic source 108 isattached to the user's hand and the tracker 114 is attached to the HMD106 if desired.

At step 404, a magnetic field is generated from the plurality ofmagnetic coils that is configured to generate a magnetic field. Whilethe magnetic field is generated, at step 406 sensor data is generated byand collected from the other plurality of magnetic coils.

At step 408, the processor generates and outputs instructions to the HMDto display on the HMD display a generated virtual world.

At step 410, the processor determines the position and orientation ofthe user's limb relative to the HMD from the sensor data.

Finally, at step 412, the processor generates and outputs instructionsto the HMD to display on the HMD a virtual limb in an apparent positionthat corresponds to the actual physical location of the user's limb.

Also as discussed above, more than one tracker 114 may be used, with theadditional trackers attached to other limbs or the torso of user 102 atstep 402. In this case, each tracker will generate sensor data at step406, and the processor will determine the position of each trackerrelative to the HMD at step 410. Finally, the processor will generateinstructions to display a virtual limb (or torso) corresponding to theactual physical position of each tracked limb or torso.

It will be apparent to one of skill in the art that some of these stepsmay be performed in a different order than specified, while othersshould be performed in order to obtain the desired effect. For example,the processor may determine the position and orientation of the trackerbefore generating and outputting the instructions to the HMD to displaythe generated virtual world. On the other hand, it will most likely notbe appropriate to send the instructions that result in the HMDdisplaying the virtual hand before the position of the tracker on theuser's physical hand has been determined.

Another possible application of the novel system and method describedherein involves tracking of a user's body parts without the use of aHMD, and thus without display of an avatar of those body parts. Byplacing a magnetic field source and trackers on the limbs and torso of auser, the system may be used for generalized motion capture bydetermining the movement of the user's limbs relative to the torso.

Such motion capture may be used as input for games other than those thatutilize a HMD, for recording performance or movements of a user to beused in 3D animation, for reactive user interfaces in graphical systems(such as was seen in the movie Minority Report), or any other field ofhuman-computer interaction that could react to the user's body pose.

Another possible application of the described system and method is incombination with traditional base station-based systems. Such acombination allows users to benefit from both the highly accurateself-relative tracking of the present application as well as trackingrelative to a fixed point (i.e., a base station) as in the prior art,while providing a mapping between the two frames of reference. Thiscould, for example, allow multiple users to share a physical and virtualspace at the same time, or allow positional tracking of a user relativeto a fixed physical location such as a floor or room.

Yet another possible application is in augmented reality situations, inwhich the user is still able to see the physical world at leastindirectly while some elements of the physical world are augmented orsupplemented by computer-generated sensory input. By contrast withvirtual reality, in which the user's perception of the physical world islargely or entirely replaced with a simulated one, in augmented realityone's perception of the physical world is not entirely replaced butrather “enhanced” with additional input.

In an augmented reality situation, a tracker may be attached to aphysical object and then tracked with respect to the user's field ofvision, making it possible to overlay computer generated graphicsprecisely on the tracked object, or alternatively to use the trackedobject to interact with virtual objects displayed to the user.

The same tracking of body parts described above with respect to avirtual world displayed on a HMD may also be applied to augmentedreality situations, without the need to render and display an avatar ofthe user. A user could thus interact with real physical objects andvirtual objects simultaneously in the same space.

In yet another possible application, tracking of a user's body couldalso be used to animate an avatar, not to be shown to the user as above,but rather to be shown to a different user viewing a separate virtualworld or an augmented reality, for use in, for example,telecommunications applications.

The disclosed system and method has been explained above with referenceto several embodiments. Other embodiments will be apparent to thoseskilled in the art in light of this disclosure. Certain aspects of thedescribed method and apparatus may readily be implemented usingconfigurations or steps other than those described in the embodimentsabove, or in conjunction with elements other than or in addition tothose described above. It will also be apparent that in some instancesthe order of steps described herein may be altered without changing theresult of performance of all of the described steps.

There may be a single processor, or multiple processors performingdifferent functions of the functions described herein. As above, aprocessor may be located in the HMD or elsewhere on the user's body, oreven in a base station if desired. One of skill in the art willappreciate how to determine which and how many processors will beappropriate for a specific intended application. Instructions forperforming the methods herein on a processor may be stored on anon-transitory computer-readable storage medium.

These and other variations upon the embodiments are intended to becovered by the present disclosure, which is limited only by the appendedclaims.

What is claimed is:
 1. A self-relative tracking system for use with ahead-mounted display (HMD), the HMD covering a user's field of vision,the HMD including a HMD tracking system for determining an absoluteposition and orientation of the HMD relative to a fixed position in thephysical world, comprising: a magnetic tracking system comprising aplurality of source magnetic coils configured to generate a magneticfield and a plurality of sensor magnetic coils configured to sense themagnetic field and generate magnetic sensor data, one of the pluralitiesof magnetic coils configured to be attached to the HMD and the otherplurality of magnetic coils configured to be attached to a limb of theuser; and a processor configured to: generate and output instructions tothe HMD to display on the HMD an augmented reality; determine a relativeposition and orientation of the users' limb relative to the HMD usingthe magnetic sensor data from the magnetic tracking system; determine anabsolute position and orientation of the user's limb relative to thefixed position in the physical world by combining the relative positionand orientation of the user's limb with the absolute position andorientation of the HMD; and generate and output instructions to the HMDto display on the HMD a virtual limb in the augmented reality, thevirtual limb in an apparent position and orientation corresponding tothe determined absolute position and orientation of the user's limb. 2.The self-relative tracking system of claim 1, wherein the magnetictracking system further comprises a second plurality of sensor magneticcoils configured to sense the magnetic field and generate additionalmagnetic sensor data, and to be attached to the torso of the user; andthe processor is further configured to: determine a relative positionand orientation of the user's torso relative to the HMD using theadditional magnetic sensor data from the magnetic tracking system;determine an absolute position and orientation of the user's torsorelative to the fixed position in the physical world by combining therelative position and orientation of the user's torso with the absoluteposition and orientation of the HMD; and generate and outputinstructions to the HMD to display on the HMD a virtual torso in theaugmented reality in an apparent position and orientation correspondingto the determined absolute position and orientation of the user's torso.3. The self-relative tracking system of claim 1, wherein the magnetictracking system further comprises a second plurality of sensor magneticcoils configured to sense the magnetic field and to be attached to aphysical object in the vicinity of the user; and the processor isfurther configured to: determine a relative position and orientation ofthe physical object relative to the HMD using the additional magneticsensor data from the magnetic tracking system; determine an absoluteposition and orientation of the physical object relative to the fixedposition in the physical world by combining the relative position andorientation of the physical object with the absolute position andorientation of the HMD; and generate and output instructions to the HMDto display on the HMD a virtual object in the augmented reality in anapparent position and orientation corresponding to the determinedabsolute position and orientation of the physical object.
 4. Theself-relative tracking system of claim 1 wherein the plurality ofmagnetic coils configured to be attached to a limb of the user iscontained in a tracking device configured to be attached to a limb ofthe user.
 5. The self-relative tracking system of claim 1 wherein theplurality of magnetic coils configured to be attached to a limb of theuser is contained in a controller configured to be held by the user. 6.The self-relative tracking system of claim 1 wherein the processor,configured to determine the absolute position and orientation of theuser's limb relative to the fixed position in the physical world bycombining the relative position and orientation of the user's limb withthe absolute position and orientation of the HMD, uses an offset betweena position of the plurality of magnetic coils attached to the HMD andthe absolute position of the HMD.
 7. The self-relative tracking systemof claim 1 wherein the plurality of source magnetic coils configured togenerate the magnetic field is the plurality of magnetic coilsconfigured to be attached to the HMD, and the plurality of sensormagnetic coils configured to sense the magnetic field and generatemagnetic sensor data is the plurality of magnetic coils configured to beattached to the limb of the user.
 8. The self-relative tracking systemof claim 1 wherein the plurality of source magnetic coils configured togenerate the magnetic field is the plurality of magnetic coilsconfigured to be attached to the limb of the user, and the plurality ofsensor magnetic coils configured to sense the magnetic field andgenerate magnetic sensor data is the plurality of magnetic coilsconfigured to be attached to the HMD.
 9. The self-relative trackingsystem of claim 1 wherein the HMD tracking system comprises an opticaltracking system.
 10. The self-relative tracking system of claim 1wherein the HMD tracking system comprises an inertial measurement unit.11. The self-relative tracking system of claim 1 wherein the HMDtracking system comprises a second magnetic tracking system comprising:a second plurality of source magnetic coils configured to generate amagnetic field; and a second plurality of sensor magnetic coilsconfigured to sense the magnetic field and generate magnetic sensordata; and wherein one of the second pluralities of magnetic coils isconfigured to be attached to the HMD and the other second plurality ofmagnetic coils is configured to be attached to the fixed, physicalworld, position.
 12. A method of providing an avatar of the body of auser wearing a head-mounted display (HMD) to the user, the HMD coveringthe user's field of vision, the HMD including a HMD tracking system fordetermining an absolute position and orientation of the HMD relative toa fixed position in the physical world, the method comprising: attachingtwo pluralities of magnetic coils to the user, one plurality of coils tothe HMD and the other plurality of magnetic coils to a limb of the user;generating a magnetic field from one of the pluralities of magneticcoils; generating magnetic sensor data by sensing the magnetic fieldusing the other plurality of coils; generating and outputting, by aprocessor, instructions to the HMD to display on the HMD an augmentedreality; determining, by the processor, a relative position andorientation of the user's limb relative to the HMD using the magneticsensor data; determining, by the processor, an absolute position andorientation of the user's limb relative to the fixed position in thephysical world by combining the relative position and orientation of theuser's limb with the absolute position and orientation of the HMD; andgenerating and outputting, by the processor, instructions to the HMD todisplay on the HMD a virtual limb in the augmented reality, the virtuallimb in an apparent position and orientation corresponding to thedetermined absolute position and orientation of the user's limb.
 13. Themethod of claim 12, further comprising: attaching a third plurality ofmagnetic coils to the torso of the user, the third plurality of magneticcoils configured to sense the magnetic field; generating additionalmagnetic sensor data by sensing the magnetic field using the thirdplurality of coils; determining, by the processor, a relative positionand orientation of the user's torso relative to the HMD using theadditional magnetic sensor data from the third plurality of coils;determining, by the processor, an absolute position and orientation ofthe user's torso relative to the fixed position in the physical world bycombining the relative position and orientation of the user's torso withthe absolute position and orientation of the HMD; and generating andoutputting, by the processor, instructions to the HMD to display on theHMD a virtual torso in the augmented reality in an apparent position andorientation corresponding to the determined absolute position andorientation of the user's torso.
 14. The method of claim 12, furthercomprising: attaching a third plurality of magnetic coils to a physicalobject in the vicinity of the user, the third plurality of magneticcoils configured to sense the magnetic field; generating additionalmagnetic sensor data by sensing the magnetic field using the thirdplurality of coils; determining, by the processor, a relative positionand orientation of the physical object relative to the HMD using theadditional magnetic sensor data from the third plurality of coils;determining, by the processor, an absolute position and orientation ofthe physical object relative to the fixed position in the physical worldby combining the relative position and orientation of the physicalobject with the absolute position and orientation of the HMD; andgenerating and outputting, by the processor, instructions to the HMD todisplay on the HMD a virtual object in the augmented reality in anapparent position and orientation corresponding to the determinedabsolute position and orientation of the physical object.
 15. The methodof claim 12, wherein determining, by the processor, the absoluteposition and orientation of the user's limb relative to the fixedposition in the physical world by combining the relative position andorientation of the user's limb with the absolute position andorientation of the HMD, uses an offset between a position of theplurality of magnetic coils attached to the HMD and the absoluteposition of the HMD.
 16. The method of claim 12, wherein: generating amagnetic field from one of the pluralities of magnetic coils comprisesgenerating the magnetic field from the plurality of magnetic coilsattached to the HMD; and generating magnetic sensor data by sensing themagnetic field using the other plurality of coils comprises sensing themagnetic field using the plurality of coils attached to the limb of theuser.
 17. The method of claim 12, wherein: generating a magnetic fieldfrom one of the pluralities of magnetic coils comprises generating themagnetic field from the plurality of magnetic coils attached to the limbof the user; and generating magnetic sensor data by sensing the magneticfield using the other plurality of coils comprises sensing the magneticfield using the plurality of coils attached to the HMD.
 18. The methodof claim 12 wherein the HMD tracking system comprises an opticaltracking system.
 19. The method of claim 12 wherein the HMD trackingsystem comprises an inertial measurement unit.
 20. The method of claim12 wherein the HMD tracking system comprises a magnetic tracking systemcomprising: a second plurality of source magnetic coils configured togenerate a magnetic field; and a second plurality of sensor magneticcoils configured to sense the magnetic field and generate magneticsensor data; and wherein one of the second pluralities of magnetic coilsis configured to be attached to the HMD and the other second pluralityof magnetic coils is configured to be attached to the fixed, physicalworld, position.
 21. A non-transitory computer readable storage mediumhaving embodied thereon instructions for causing a computing device toexecute a method of providing an avatar of the body of a user wearing ahead-mounted display (HMD) to the user, the HMD covering the user'sfield of vision, the HMD including a HMD tracking system for determiningan absolute position and orientation of the HMD relative to a fixedposition in the physical world, the method comprising: generating andoutputting, by a processor, instructions to the HMD to display on theHMD an augmented reality; determining, by the processor, a relativeposition and orientation of the user's limb relative to the HMD usingmagnetic sensor data generated by one plurality of magnetic coils,attached to a limb of the user, sensing a magnetic field generated byanother plurality of magnetic coils attached to the HMD; determining, bythe processor, an absolute position and orientation of the user's limbrelative to the fixed position in the physical world by combining therelative position and orientation of the user's limb with the absoluteposition and orientation of the HMD; and generating and outputting, bythe processor, instructions to the HMD to display on the HMD a virtuallimb in the augmented reality, the virtual limb in an apparent positionand orientation corresponding to the determined absolute position andorientation of the user's limb.